Collaborative training of heterogeneous reinforcement learning agents in environments with sparse rewards: what and when to share?

In the early stages of human life, babies develop their skills by exploring different scenarios motivated inherent satisfaction rather than extrinsic rewards from environment. This behavior, referred to as intrinsic motivation, has emerged one solution address exploration challenge derived reinforcement learning environments with sparse rewards. Diverse approaches have been proposed accelerate process over single- and multi-agent problems homogeneous agents. However, scarce studies elaborated on collaborative frameworks between heterogeneous agents deployed into same environment, but interacting instances latter without any prior knowledge. Beyond heterogeneity, each agent’s characteristics grant access only a subset full state space, which may hide strategies optimal solutions. this work we combine ideas motivation transfer learning. Specifically, focus sharing parameters in actor-critic model architectures combining information obtained through aim having more efficient faster We test our experiments performed modified ViZDooM’s My Way Home scenario, is challenging its original version allows evaluating heterogeneity Our results reveal ways framework little additional computational cost can outperform an independent knowledge sharing. Additionally, depict need for modulating correctly importance avoid undesired agent behaviors.